datadog_checks_base: Fix obfuscate_sql_with_metadata wrapper not handling json.loads() edge case #11038

alexbarksdale · 2022-01-05T22:09:34Z

What does this PR do?

We found that the json decoder (json.loads()) does not throw an error for a string integer (e.g. '1'). This causes unwanted behavior and this PR aims to fix that.

How?
Because we know we only want to load json when a particular option is enabled, we use that to determine the action rather than always decoding.

Motivation

Additional Notes

Review checklist (to be filled by reviewers)

Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
PR title must be written as a CHANGELOG entry (see why)
Files changes must correspond to the primary purpose of the PR as described in the title (small unrelated changes should have their own PR)
PR must have changelog/ and integration/ labels attached

codecov · 2022-01-05T22:16:03Z

Codecov Report

Merging #11038 (b376646) into master (5a1a7f9) will increase coverage by 0.23%.
The diff coverage is 100.00%.

Flag	Coverage Δ
active_directory	`100.00% <ø> (+17.64%)`	⬆️
activemq_xml	`82.31% <ø> (ø)`
aerospike	`86.97% <ø> (+0.36%)`	⬆️
airflow	`90.00% <ø> (ø)`
amazon_msk	`88.83% <ø> (ø)`
ambari	`85.75% <ø> (ø)`
apache	`95.08% <ø> (ø)`
aspdotnet	`100.00% <ø> (+26.19%)`	⬆️
avi_vantage	`91.92% <ø> (ø)`
azure_iot_edge	`82.00% <ø> (ø)`
btrfs	`82.91% <ø> (ø)`
cacti	`83.95% <ø> (ø)`
cassandra_nodetool	`94.19% <ø> (ø)`
ceph	`91.02% <ø> (ø)`
cilium	`89.74% <ø> (+1.02%)`	⬆️
cisco_aci	`95.83% <ø> (ø)`
citrix_hypervisor	`87.50% <ø> (ø)`
clickhouse	`95.63% <ø> (ø)`
cloud_foundry_api	`95.98% <ø> (+0.12%)`	⬆️
cockroachdb	`100.00% <ø> (ø)`
consul	`91.74% <ø> (ø)`
coredns	`95.74% <ø> (ø)`
couch	`95.19% <ø> (+0.24%)`	⬆️
couchbase	`83.46% <ø> (ø)`
crio	`89.79% <ø> (ø)`
datadog_checks_base	`90.30% <100.00%> (+0.37%)`	⬆️
datadog_checks_dev	`80.14% <ø> (-0.70%)`	⬇️
datadog_checks_downloader	`80.64% <ø> (ø)`
datadog_cluster_agent	`90.00% <ø> (ø)`
directory	`94.87% <ø> (ø)`
disk	`91.61% <ø> (ø)`
dns_check	`93.84% <ø> (ø)`
dotnetclr	`94.93% <ø> (+15.18%)`	⬆️
druid	`97.70% <ø> (ø)`
ecs_fargate	`80.23% <ø> (ø)`
eks_fargate	`94.05% <ø> (ø)`
elastic	`90.52% <ø> (ø)`
envoy	`94.25% <ø> (ø)`
etcd	`93.87% <ø> (ø)`
exchange_server	`98.29% <ø> (+12.82%)`	⬆️
external_dns	`89.09% <ø> (ø)`
fluentd	`94.77% <ø> (ø)`
gearmand	`78.26% <ø> (+1.24%)`	⬆️
gitlab	`89.94% <ø> (ø)`
gitlab_runner	`91.94% <ø> (ø)`
glusterfs	`80.09% <ø> (+0.92%)`	⬆️
go_expvar	`92.73% <ø> (ø)`
gunicorn	`93.60% <ø> (+0.75%)`	⬆️
haproxy	`95.09% <ø> (+0.16%)`	⬆️
harbor	`81.29% <ø> (ø)`
hazelcast	`92.39% <ø> (ø)`
hdfs_datanode	`89.74% <ø> (ø)`
hdfs_namenode	`86.72% <ø> (ø)`
http_check	`90.98% <ø> (+1.74%)`	⬆️
ibm_db2	`94.84% <ø> (ø)`
ibm_i	`80.65% <ø> (ø)`
ibm_mq	`89.67% <ø> (ø)`
ibm_was	`96.06% <ø> (ø)`
iis	`94.91% <ø> (+38.49%)`	⬆️
istio	`77.46% <ø> (+0.56%)`	⬆️
kafka_consumer	`82.66% <ø> (ø)`
kong	`88.58% <ø> (ø)`
kube_apiserver_metrics	`97.67% <ø> (ø)`
kube_controller_manager	`96.85% <ø> (ø)`
kube_dns	`98.85% <ø> (ø)`
kube_metrics_server	`100.00% <ø> (ø)`
kube_proxy	`100.00% <ø> (ø)`
kube_scheduler	`96.20% <ø> (ø)`
kubelet	`89.61% <ø> (ø)`
kyototycoon	`85.96% <ø> (ø)`
lighttpd	`83.64% <ø> (ø)`
linkerd	`85.14% <ø> (+1.14%)`	⬆️
linux_proc_extras	`96.22% <ø> (ø)`
mapr	`82.70% <ø> (ø)`
mapreduce	`81.30% <ø> (-0.47%)`	⬇️
marathon	`83.12% <ø> (ø)`
marklogic	`95.33% <ø> (ø)`
mcache	`93.52% <ø> (ø)`
mesos_master	`90.68% <ø> (ø)`
mesos_slave	`93.63% <ø> (ø)`
mysql	`87.21% <ø> (+0.16%)`	⬆️
nagios	`89.53% <ø> (ø)`
network	`77.76% <ø> (+1.00%)`	⬆️
nfsstat	`95.20% <ø> (ø)`
nginx	`95.27% <ø> (+0.55%)`	⬆️
nginx_ingress_controller	`98.36% <ø> (ø)`
openldap	`96.33% <ø> (ø)`
openmetrics	`97.14% <ø> (ø)`
openstack	`51.45% <ø> (ø)`
openstack_controller	`90.74% <ø> (ø)`
oracle	`92.58% <ø> (+2.69%)`	⬆️
pdh_check	`95.65% <ø> (ø)`
pgbouncer	`90.45% <ø> (ø)`
php_fpm	`90.25% <ø> (+0.84%)`	⬆️
postfix	`88.04% <ø> (ø)`
postgres	`91.55% <ø> (+0.21%)`	⬆️
powerdns_recursor	`96.65% <ø> (ø)`
process	`85.07% <ø> (ø)`
prometheus	`94.17% <ø> (ø)`
proxysql	`98.97% <ø> (ø)`
rabbitmq	`94.40% <ø> (ø)`
redisdb	`87.44% <ø> (ø)`
rethinkdb	`97.93% <ø> (ø)`
riak	`99.22% <ø> (ø)`
riakcs	`93.61% <ø> (ø)`
scylla	`100.00% <ø> (ø)`
singlestore	`90.81% <ø> (ø)`
snmp	`87.96% <ø> (-0.51%)`	⬇️
snowflake	`93.60% <ø> (ø)`
sonarqube	`95.69% <ø> (ø)`
spark	`93.55% <ø> (+0.04%)`	⬆️
sqlserver	`84.27% <ø> (+0.19%)`	⬆️
squid	`100.00% <ø> (ø)`
ssh_check	`91.58% <ø> (ø)`
statsd	`87.36% <ø> (+1.05%)`	⬆️
supervisord	`92.30% <ø> (ø)`
system_core	`91.04% <ø> (ø)`
system_swap	`98.30% <ø> (ø)`
tcp_check	`89.83% <ø> (ø)`
teamcity	`80.00% <ø> (ø)`
tls	`97.04% <ø> (+0.87%)`	⬆️
tokumx	`58.40% <ø> (?)`
twemproxy	`78.33% <ø> (ø)`
twistlock	`79.62% <ø> (ø)`
varnish	`84.57% <ø> (+0.24%)`	⬆️
vault	`95.04% <ø> (+0.55%)`	⬆️
vertica	`92.33% <ø> (ø)`
voltdb	`96.81% <ø> (ø)`
vsphere	`89.78% <ø> (+0.08%)`	⬆️
win32_event_log	`86.32% <ø> (+0.56%)`	⬆️
windows_performance_counters	`98.36% <ø> (ø)`
windows_service	`95.83% <ø> (ø)`
wmi_check	`92.91% <ø> (ø)`
yarn	`89.85% <ø> (ø)`
zk	`86.04% <ø> (+0.46%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

remeh

Not sure why agent-core has to review this PR.
Code change LGTM except for a couple remarks, please make sure to get a review from integrations-core team.

datadog_checks_base/datadog_checks/base/utils/db/utils.py

JSON decoder fix

7134586

alexbarksdale requested review from a team as code owners January 5, 2022 22:09

ghost added the integration/datadog_checks_base label Jan 5, 2022

alexbarksdale added the changelog/no-changelog label Jan 5, 2022

alexandre-normand previously approved these changes Jan 5, 2022

View reviewed changes

Cleanup

b376646

alexbarksdale dismissed alexandre-normand’s stale review via b376646 January 6, 2022 04:17

remeh approved these changes Jan 6, 2022

View reviewed changes

datadog_checks_base/datadog_checks/base/utils/db/utils.py Show resolved Hide resolved

datadog_checks_base/datadog_checks/base/utils/db/utils.py Show resolved Hide resolved

alexandre-normand approved these changes Jan 6, 2022

View reviewed changes

fanny-jiang approved these changes Jan 6, 2022

View reviewed changes

alexbarksdale merged commit 0bc9b53 into master Jan 6, 2022

alexbarksdale deleted the alex.barksdale/fix-obfuscate-wrapper-json-decoder branch January 6, 2022 16:22

yzhan289 added changelog/Fixed and removed changelog/no-changelog labels Jan 7, 2022

yzhan289 mentioned this pull request Jan 7, 2022

Bump base package #11067

Merged

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

datadog_checks_base: Fix obfuscate_sql_with_metadata wrapper not handling json.loads() edge case #11038

datadog_checks_base: Fix obfuscate_sql_with_metadata wrapper not handling json.loads() edge case #11038

alexbarksdale commented Jan 5, 2022 •

edited

Loading

codecov bot commented Jan 5, 2022 •

edited

Loading

remeh left a comment •

edited

Loading

datadog_checks_base: Fix obfuscate_sql_with_metadata wrapper not handling json.loads() edge case #11038

datadog_checks_base: Fix obfuscate_sql_with_metadata wrapper not handling json.loads() edge case #11038

Conversation

alexbarksdale commented Jan 5, 2022 • edited Loading

What does this PR do?

Motivation

Additional Notes

Review checklist (to be filled by reviewers)

codecov bot commented Jan 5, 2022 • edited Loading

Codecov Report

remeh left a comment • edited Loading

Choose a reason for hiding this comment

alexbarksdale commented Jan 5, 2022 •

edited

Loading

codecov bot commented Jan 5, 2022 •

edited

Loading

remeh left a comment •

edited

Loading